Imaging apparatus, imaging processing method, image processing device and imaging processing system

ABSTRACT

An imaging apparatus includes: an image sensor; a signal processing unit repeatedly performing processing different from each other in a given number of sequential frames to imaging signals obtained by the image sensor to thereby obtain image data of respective frame; and an image data output unit sequentially outputting the image data of respective frames.

FIELD

The present disclosure relates to an imaging apparatus, an imagingprocessing method, an image processing device and an imaging processingsystem, and particularly relates to an imaging apparatus and so onsuitable to be applied to an in-vehicle camera and the like.

BACKGROUND

An in-vehicle camera arranged at the back of a vehicle and the like toobtain a taken image of the vehicle periphery is known as related art(refer to JP-A-2002-330428 (Patent Document 1)). The in-vehicle cameraoutputs wide-angle image data corresponding to, for example, visualimage data. To obtain particular image data such as bird's eye-viewconversion image data and reduced-screen image data is performed by acamera ECU in a subsequent stage and so on, which increases theprocessing load of the camera ECU.

SUMMARY

It is desirable to facilitate processing of the camera ECU.

An embodiment of the present disclosure is directed to an imagingapparatus including an image sensor, a signal processing unit repeatedlyperforming processing different from each other in a given number ofsequential frames to imaging signals obtained by the image sensor tothereby obtain image data of respective frames, and an image data outputunit sequentially outputting the image data of respective frames.

In the embodiment of the present disclosure, imaging is performed by theimage sensor to obtain imaging signals. The processing different fromeach other in the given number of sequential frames is repeatedlyperformed to the imaging signals by the signal processing unit tothereby obtain image data of respective frames. Then, image data ofrespective frames is outputted by the image data output unit.

It is possible, for example, that the signal processing unit obtainsfirst frame image data by performing first processing in a first frameand obtains second frame image data by performing second processing in asecond frame in sequential two frames of the imaging signals obtained bythe image sensor. In this case, the first frame image data may be visualimage data and the second frame image data may be image data to whichbird's eye-view conversion is performed. Additionally, in this case, theimage data to which the bird's eye-view conversion is performed may beimage data obtained by superimposing a chroma edge signal on a luminancesignal.

Furthermore, in this case, the first frame image data may be normalimage data and the second frame image data may be image data obtained bysuperimposing a chroma edge signal on a luminance signal. Moreover, inthis case, the first image data may be normal image data and the secondimage data may be sensor image data of an electronic shutter or datadifferent in an exposure accumulation method. Also in this case, thefirst frame image data may be image data of a whole screen and thesecond image data is image data of a reduced screen. Also in this case,the first frame image data may be color image data and the second frameimage data may be monochrome image data. The image data may also beobtained by performing processing at random to part of frames insequential frames, not repeatedly performing processing alternately.

As described above, according to the embodiment of the presentdisclosure, frame image data obtained by performing a given number of,for example, two different processing can be sequentially outputted.Accordingly, the camera ECU in the subsequent stage can acquire theseimage data without performing processing of obtaining particular imagedata such as camera bird's eye-view conversion image data andreduced-screen image data. Accordingly, the processing load of thecamera ECU can be reduced and the processing can be facilitated.

In the embodiment of the present disclosure, it is possible that, forexample, the imaging apparatus further includes a header addition unitadding a header at least including identification information foridentifying to which frame the image data corresponds in the givennumber of frames to image data of each frame obtained by the signalprocessing unit. As the header is added to image data of each frame asdescribed above, for example, the camera ECU in the subsequent stage candetermine information of the frame to be taken concerning what type ofprocessing has been performed to image data of each frame accurately andeasily at the time of starting processing of taking frames, therefore,the processing of taking frames can be appropriately performedselectively.

In this case, the header addition unit may add the header to an areaoutside an effective image area or an area inside the effective imagearea in each frame. As the header is added to the area inside theeffective image area, for example, the header information can beadequately supplied to the camera ECU even in a case where the cameraECU in the subsequent stage takes only the effective image area. Forexample, the header addition unit may add the header by using part ofbits in a given number of pixel data when adding the header in the areainside the effective image area in each frame. Accordingly, it ispossible to suppress effects to image data due to the addition of theheader to the area inside the effective image area.

Another embodiment of the present disclosure is directed to an imagingapparatus including an image sensor, and an information addition unitadding information of peculiar setting and processing to a headerportion of a corresponding frame as identification determinationinformation in accordance with signal processing of an imaging signalobtained by the image sensor.

Still another embodiment of the present disclosure is directed to aimaging apparatus including an image sensor, and an image processingunit, in which the image processing unit extracts a chroma edge signalwith respect to part of a taken image obtained by the image sensor andsuperimposes the chroma edge signal on a luminance signal not having aluminance signal boundary in the same direction.

Yet another embodiment of the present disclosure is directed to an imageprocessing device including an image data input unit inputting imagedata, in which the image data is obtained by repeatedly performingprocessing different from each other in a given number of sequentialframes or at random, and a header at least including identificationinformation for identifying to which frame the image data corresponds inthe given number of frames is added to image data of each frame, and aprocessing unit appropriately performing processing of the inputtedimage data of each frame based on the added header.

Still yet another embodiment of the present disclosure is directed to animaging processing system including an imaging apparatus, and an imageprocessing device, in which the imaging apparatus includes an imagesensor, a signal processing unit repeatedly performing processingdifferent from each other in a given number of sequential frames toimaging signals obtained by the image sensor to thereby obtain imagedata of respective frames, a header addition unit adding a header atleast including identification information for identifying to whichframe the image data corresponds in the given number of frames to theimage data of each frame, and an image data output unit outputting theimage data of respective frames, and in which the image processingdevice includes an image data input unit inputting image data, and aprocessing unit appropriately performing processing of the inputtedimage data of each frame based on the added header.

In the embodiment of the present disclosure, the header may include atleast one or more information of sensor electronic shutter, sensoramplifier gain and lens aperture information, lens filter information,temperature information, image differentiation component adjustmentparameter values, image gray-scale adjustment parameters, imageresolution adjustment parameters, processing between frames adjustmentparameters, illumination estimation information, environment estimationinformation, camera AE adjustment information, camera WB adjustmentinformation, image conversion presence information, image histograminformation, image conversion setting information and image cutoutcoordinate information.

Further another embodiment of the present disclosure is directed to animaging processing system having an imaging apparatus and an imageprocessing device, including a means for outputting images in units offrames by the imaging apparatus, a means for adding a means for uniquelyidentifying the frame to a header of the image to be outputted, and aselection means for appropriately allocating the outputted images tounique memory addresses based on the image headers.

According to the embodiments of the present disclosure, it is possibleto facilitate processing of the camera ECU.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration example of an imagingprocessing system as an embodiment;

FIG. 2A is a view schematically showing a positional relationshipbetween image data of one frame and a header to be added thereto, andFIG. 2B is a view showing a configuration example of a pixel dataportion to which the header is added;

FIG. 3 is a view showing an example in which frame image data of avisual image (normal wide-angle camera image) and frame image data of abird's eye-view conversion image are alternately outputted;

FIG. 4 is a flowchart showing an example of a processing procedure of anin-vehicle camera included in the imaging processing system;

FIGS. 5A to 5C are views showing an example of obtaining image data of abird's eye-view conversion image from image data of a road surface areaas a part of a whole screen by performing bird's eye-view imageconversion;

FIG. 6 is an example of a frame image (before superimposition) forexplaining processing of superimposing a chroma edge signal on aluminance signal;

FIG. 7 is an example of a frame image (after superimposition) forexplaining the processing of superimposing the chroma edge signal on theluminance signal in an image of a given area;

FIG. 8 is a flowchart showing an example of a processing procedure of acamera ECU included in the imaging processing system;

FIG. 9 is a flowchart showing another example of a processing procedureof the camera ECU included in the imaging processing system;

FIGS. 10A to 10C are views showing an example of a picture-in-pictureimage in which a bird's eye-view conversion image near the back of avehicle is inserted into a normal wide-angle image of the vehicleperiphery;

FIG. 11 is a view showing an example in which frame image data of awhole screen and frame image data of a reduced screen are alternatelyoutputted;

FIG. 12 is a flowchart showing another example of a processing procedureof the in-vehicle camera included in the imaging processing system;

FIG. 13 is a flowchart showing another example of a processing procedureof the camera ECU included in the imaging processing system; and

FIG. 14 is a view showing an example in which frame image data of acolor image and frame image data of a monochrome image are alternatelyoutputted.

DETAILED DESCRIPTION

Hereinafter, a mode for carrying out the present disclosure (hereinafterreferred to as an “embodiment”) will be explained. The explanation willbe made in the following order.

1. Embodiment

2. Modification Example

<1. Embodiment>

[Configuration Example of Imaging Processing System]

FIG. 1 shows a configuration example of an imaging processing system 10as an embodiment. The imaging processing system 10 includes anin-vehicle camera 100 arranged at the back of a vehicle and the like andtaking an image of the vehicle periphery, a camera ECU (electroniccontrol unit) 200 having an image processing function and a monitor 300.

The in-vehicle camera 100 configures an imaging apparatus. Thein-vehicle camera 100 includes a control unit 101, a lens unit 102, asensor unit 103, an imaging signal processing unit 104, a postprocessing unit 105 and an output unit 106. The in-vehicle camera 100and the camera ECU 200 are connected by a cable 400. The control unit101 controls operations of respective units of the in-vehicle camera100.

The lens unit 102 forms an object image on an imaging surface of thesensor unit 103. The sensor unit 103 obtains an imaging signal based onthe object image formed on the imaging surface. The imaging signalprocessing unit 104 performs AE (Automatic Exposure) adjustment, AWB(Automatic White Balance) adjustment and so on to the imaging signalobtained by the sensor unit 103 to obtain taken image data. The postprocessing unit 105 performs various processing to the taken image dataobtained in the imaging signal processing unit 104 to obtain image datato be transmitted to the camera ECU 200. The output unit 106 transmitsthe image data obtained in the post processing unit 105 to the cable400.

The post processing unit 105 adds a header including setting(adjustment) information, environmental information and the like ofrespective units of the in-vehicle camera 100 to image data in eachframe. Here, the environmental information is information ofenvironmental light, environmental temperature and so on. In this case,the post processing unit 105 adds the header to either of an areaoutside an effective image area such as a blanking period or an areainside the effective image area. In the following description of theembodiment, the header is assumed to be added to the area inside theeffective image area. As the header is added to the area inside theeffective image area, header information can be appropriately suppliedto the camera ECU 200 even when the camera ECU 200 in the subsequentstage takes only the area inside the effective image area.

FIG. 2A schematically shows a positional relationship between image dataof one frame and the header to be added thereto. The header is added tothe head of the image data. The drawing shows an example in which datais added to the front half of the first line. It is also preferable thatdata is added to the entire line, and further added to several lines.FIG. 2B shows a configuration example of a pixel data portion to whichthe header is added. As shown in the example, when pixel data is formedby N bits, for example, only low-order M bits may be allocated to theheader. Accordingly, it is possible to suppress effects to image datadue to the addition of header in the area inside the effective imagearea.

As described above, the header includes peculiar various setting(adjustment) information of respective units of the in-vehicle camera100 in each frame. It is not necessary to include all the setting(adjustment) information and at least necessary information in thecamera ECU 200 in the subsequent stage may be included. As setting(adjustment) information of the lens unit 102, for example, there areaperture setting information, filter setting information and so on.

As setting (adjustment) information of the sensor unit 103, for example,there are electronic shutter information, reading-operation settinginformation, addition setting information, amplifier gain settinginformation, filter arrangement information, sensitivity variationinformation, lens mount displacement information and so on. As setting(adjustment) information of the imaging signal processing unit 104, forexample, there are AE adjustment information, AWB adjustmentinformation, environment estimation information, output image sizeinformation, image cutout coordinate information, image conversioninformation, image right/left and up/down inversion information, imageemphasis processing application value information, application noisereduction processing information, image histogram information and so on.

The post processing unit 105 also repeatedly performs processingdifferent from each other in a given number of sequential frames torespective image data obtained in the imaging signal processing unit 104to thereby obtain image data of respective frames to be transmitted. Inthe embodiment, first frame image data is obtained by performing firstprocessing in the first frame, and second frame image data is obtainedby performing second processing in the second frame in a sequential twoframes.

The above-described header added to each frame also includesidentification information for identifying to which frame the image datacorresponds. The first processing and the second processing will bedescribed later. The example in which processing is alternatelyperformed to the images in the first frame and the second frame will beexplained in the embodiment, however, the second processing may beperformed at random to images continuously outputted from the camera, ordifferent processing may be combined to be performed.

The camera ECU 200 includes a control unit 201, an input unit 202 and animage signal processing unit 203. The control unit 201 controlsoperations of respective units of the camera ECU 200. The input unit 202takes image data transmitted from the in-vehicle camera 100 from thecable 400. The image signal processing unit 203 appropriately processesimage data of each frame based on the header added to the image data tothereby obtain image data for display and so on. The image data fordisplay is supplied to the monitor 300. A specific example of processingin the image signal processing unit 203 will be described later.

The operation of the imaging processing system 10 shown in FIG. 1 willbe briefly explained. In the in-vehicle camera 100, an imaging signalobtained in the sensor unit 103 is supplied to the imaging signalprocessing unit 104. In the imaging signal processing unit 104, AEadjustment, AWB adjustment and so on are performed to the imaging signalto obtain taken image data. The taken image data is supplied to the postprocessing unit 105. In the post processing unit 105, various processingis performed to the taken image data to obtain image data to betransmitted to the camera ECU 200. The image data is transmitted fromthe output unit 106 to the cable 400.

Here, in the post processing unit 105, the header including setting(adjustment) information, environment information and the like ofrespective units of the in-vehicle camera 100 is added to the image dataof each frame. Also in the post processing unit 105, the firstprocessing is performed in the first frame in sequential two frames toobtain the first frame image data, and the second processing isperformed in the second frame to obtain the second frame image data. Theheader added to each frame also includes identification information foridentifying to which frame the image data corresponds. Though theexample of two-frame alternate output is shown in the embodiment, outputmay be performed repeatedly in more number of frames. A configuration ofperforming sporadic and random peculiar processing may also be applied.

In the camera ECU 200, image data transmitted from the in-vehicle camera100 is taken into the input unit 202 from the cable 400. The image datais supplied to the image signal processing unit 203. In the image signalprocessing unit 203, image data in each frame is appropriately processedthrough a selection circuit based on the header added to the image dataeven not through the control unit 201, thereby obtaining image data fordisplay and the like. The image data for display is supplied to themonitor 300, and given image data is displayed on the monitor 300. Inthis case, image data of respective frames is appropriately allocated tounique memory addresses in accordance with the image headers andprocessed.

SPECIFIC EXAMPLES OF PROCESSING IN IN-VEHICLE CAMERA AND CAMERA ECUSpecific Example 1

In this example, the in-vehicle camera 100 alternately outputs frameimage data of a visual image (normal wide-angle camera image) 1 a andframe image data of a bird's eye-view conversion image 1 b as shown inFIG. 3.

A flowchart of FIG. 4 shows an example of a processing procedure in thein-vehicle camera 100. The in-vehicle camera 100 starts processing inStep ST1. Next, in Step ST2, the sensor unit 103 changes a cutout areain alternate frames and outputs imaging signals with flags. For example,the cutout area in an odd-number frame is a whole screen and the cutoutarea in an even-number frame is a road surface area. For example, FIG.5A shows an example of the whole screen and FIG. 5B in broken linesshows an example of the road surface area.

Next, in Step ST3, the imaging signal processing unit 104 takes theimaging signal from the sensor unit 103. The imaging signal processingunit 104 determines whether the frame is the odd-number frame in whichthe cutout area is the whole screen or the even-number frame in whichthe cutout area is the road surface area based on the added flag in StepST4.

When the frame is the odd-number frame in which the cutout area is thewhole screen, the imaging signal processing unit 104 performs ISPprocessing such as AE/AWB for the normal visual camera in Step ST5.Then, the post processing unit 105 performs image-quality improvementprocessing for improving visibility such as contrast emphasis in StepST6. The post processing unit 105 also allows the header to includeidentification information indicating that the frame image data is avisual image in Step ST7.

When the frame is the even-number frame in which the cutout area is theroad surface area in Step ST4, the imaging signal processing unit 104performs given processing to the imaging signal taken from the sensorunit 103 and sends the signal to the post processing unit 105, and thepost processing unit 105 performs processing of superimposing a chromaedge signal on a luminance signal.

As an effect of improving the accuracy with respect to the calculatedamount performed in a color image is limited in many cases in imagerecognition processing, a detection method of using only the luminancesignal is often used. However, in an application for recognizing whitelines during driving, when yellow marks in a no-overtaking lane paintedon a concrete road are represented in a gray-scale image, there arises aproblem that luminance difference in concrete is small and thus it isdifficult to recognize the marks as lane boundaries.

The post processing unit 105 extracts, for example, a differentiationchange component of a color-difference signal which is a chroma signalfrom an image signal of a luminance color-difference signal andsuperimposes the signal on the luminance signal. Accordingly, it ispossible to draw boundaries in the luminance signal and to realizeprocessing of image recognition only based on analysis of the luminancesignal.

Here, the processing of superimposing the chroma edge signal on theluminance signal will be explained with reference to a frame image shownin FIG. 6. In the frame image, portions denoted by numerals 1, 2 and 3indicate yellow marks in the no-overtaking lane painted on the concreteroad. Portions denoted by numerals 4, 5 and 6 indicate white linespainted on the concrete road. The post processing unit 105 allows thechroma signal to transmit through a boundary detection filter only in anarea used for recognition shown by being surrounded by a broken line inFIG. 7, and superimposes an output value on the luminance signal toperform output as denoted by a numeral 8. In this case, the chroma edgesignal is extracted and is superimposed on the luminance signal nothaving a luminance signal boundary in the same direction.

Return to FIG. 4, after performing processing of superimposing thechroma edge signal on the luminance signal in Step ST8, the postprocessing unit 105 performs bird's eye-view image conversion in StepST9. In this case, the post processing unit 105 performs bird's eye-viewimage conversion of the image data in the road surface area shown inFIG. 5B to obtain image data of a bird's eye-view conversion image shownin FIG. 5C. The details of the method of generating the bird's eye-viewconversion image are described in, for example, the transactions of theInstitute of Electrical Engineers IP-08-21 IIS-08-46. Subsequently, thepost processing unit 105 allows the header to include identificationinformation indicating that the frame image data is the bird's eye-viewconversion image in Step ST10. In this case, the header also includesarea information of the area used for recognition, in which the chromaedge signal is superimposed on the luminance signal.

Next, in Step in ST11, the output unit 106 integrates two frames andoutputs the frame to the cable 400. That is, when the frame is anodd-number frame, image data of the visual image obtained in the postprocessing unit 105 is outputted, and when the frame is an even-numberframe, image data of the bird's eye-view conversion image obtained inthe post processing unit 105 is outputted.

Next, the in-vehicle camera 100 determines whether the processing endsor not in Step ST12. When the processing does not end, the processreturns to Step ST2 and the same process described above is repeated.When the processing ends, the processing is ended immediately in StepS13.

A flowchart of FIG. 8 shows an example of a processing procedure of thecamera ECU 200. The flowchart shows the processing procedure withrespect to one frame image data. The camera ECU 200 starts processing inStep ST21. Next, in Step ST22, the input unit 202 takes one frame ofimage data transmitted from the in-vehicle camera 100.

Next, in Step ST23, the image signal processing unit 203 extracts aheader continued from a synchronization signal of the image data. Then,the image signal processing unit 203 determines whether the frame is thevisual image frame or the bird's eye-view conversion image frame basedon frame information included in the header in Step ST24.

When the frame is not the visual image frame, namely, the frame is thebird's eye-view conversion image frame, the image signal processing unit203 appropriately performs white-line recognition processing withrespect to image information including only a luminance value of theframe image data in Step ST25, and extracts white-line coordinates inStep ST26. Then, the image signal processing unit 203 delivers theextracted white-line coordinate data with a time stamp to a vehiclecontrol ECU (not shown in FIG. 1) in Step ST27.

Here, the image signal processing unit 203 can estimate a white-linesearch area in advance and perform search processing suitable forrecognizing nighttime/daytime and so on by using advance informationsuch as recognition area information and environmental light included inthe header, which can reduce searching time and improve the accuracy ofsearching.

Next, the camera ECU 200 determines whether the process ends or not inStep ST29. When the process does not end, the process returns to StepST22 and proceeds with processing of next one frame. On the other hand,when it is determined that the process ends, the process is ended inStep ST30.

On the other hand, when the frame is the visual image frame, the imagesignal processing unit 203 performs visibility improvement processingsuch as contrast emphasis processing, edge emphasis processing to theframe image data and output the data to the monitor 300 in Step ST28.Accordingly, a normal wide-angle camera image of the vehicle peripherytaken by the in-vehicle camera 100 is displayed on the monitor 300. Forexample, a monitor of a navigation system and so on is used also as themonitor 300, though not described above.

As described above, image data of an image for a visual purpose (visualimage) and image data of an image for a white-line recognition purpose(bird's eye-view conversion image) are alternately outputted from thein-vehicle camera 100. Accordingly, it is possible to selectivelyperform the white-line recognition processing to the image data of thebird's eye-view conversion image without interposing the control unit201 for determining switching of data in the camera ECU 200 in thesubsequent stage, as a result, it is not necessary that the control unit201 determines a destination of image processing and the processing loadcan be drastically reduced.

A flowchart of FIG. 9 shows an example of another processing procedureof the camera ECU 200. The flowchart shows the processing procedure withrespect to one frame image data. The camera ECU 200 starts processing inStep ST61. Next, in Step ST62, the input unit 202 takes one frame ofimage data transmitted from the in-vehicle camera 100.

Next, in Step ST63, the image signal processing unit 203 extracts aheader continued from a synchronization signal of the image data. Then,the image signal processing unit 203 determines whether the frame is thevisual image frame or the bird's eye-view conversion image frame basedon frame information included in the header in Step ST64.

When the frame is the visual image frame, the image signal processingunit 203 writes image data in a whole display setting area of themonitor (except a later-described P in P area) in a display memory inStep ST65. On the other hand, when the frame is the bird's eye-viewconversion image frame, the image signal processing unit 203 writesimage data of the bird's eye-view conversion image in apicture-in-picture (P in P) area in the display memory after performingdown-scaling processing if necessary in Step ST66. Then, the imagesignal processing unit 203 outputs image data in the display memory tothe monitor 300 in Step ST67.

Next, the camera ECU 200 determines whether the process ends or not inStep ST68. When the process does not end, the process returns to StepST62, and proceeds with processing of next one frame. On the other hand,when it is determined that the process ends, the process is ended inStep ST69.

According to the processing of the flowchart of FIG. 9, apicture-in-picture image in which the bird's eye-view conversion imagenear the back of the vehicle is inserted into the normal wide-angleimage of the vehicle periphery is displayed on the monitor 300. Forexample, when FIG. 10A is a normal wide-angle image and FIG. 10B is abird's eye-view conversion image, a picture-in-picture image shown inFIG. 10C is displayed on the monitor 300.

As described above, image data of an image for the visual purpose(visual image) and image data of the image for the white-linerecognition purpose (bird's eye-view conversion image) are alternatelyoutputted from the in-vehicle camera 100, therefore, it is possible todisplay the picture-in-picture image using both image data in the cameraECU 200 in the subsequent stage.

Specific Example 2

In this example, the in-vehicle camera 100 alternately outputs frameimage data of a whole screen 1 d and frame image data of a reducedscreen 1 c as shown in FIG. 11. In the example, the frame image data ofthe reduced image 1 c is a ¼ pixel-reduced screen obtained by performingneighboring pixel addition for giving priority to sensitivity.

A flowchart of FIG. 12 shows another example of a processing procedureof the in-vehicle camera 100. The in-vehicle camera 100 startsprocessing in Step ST31. Next, in Step ST32, the sensor unit 103 changesthe screen to the whole screen or the reduced screen in alternate framesand outputs imaging signals with flags. For example, the odd-numberframe is the whole screen and the even-number frame is the ¼ reducedscreen.

Next, in Step ST33, the imaging signal processing unit 104 takes theimaging signal from the sensor unit 103. The imaging signal processingunit 104 determines whether the frame is the odd-number frame of thewhole screen or the even-number frame of the ¼ reduced screen based onthe added flag in Step ST34.

When the frame is the odd-number frame of the whole screen, the imagingsignal processing unit 104 performs given processing to the imagingsignal taken from the sensor unit 103 and outputs the signal to the postprocessing unit 105. The post processing unit 105 performs image-qualityimprovement processing for giving priority to daytime resolution in StepST35. The post processing unit 105 also allows the header to includeidentification information indicating that the frame image data is thewhole screen in Step ST36.

When the frame is determined to be the even-number frame of the ¼reduced screen in Step ST34, the imaging signal processing unit 104performs reduced memory allocation (4-frames cyclic allocation) in StepST37. Then, the imaging signal processing unit 104 and the postprocessing unit 105 perform priority processing for dark-timesensitivity in Step ST38. For example, frame addition processingeffectively using a memory area and three-dimensional noise reductionprocessing are performed.

Additionally, the post processing unit 105 allows the header to includeidentification information indicating that the frame image data is the ¼reduced screen in Step ST39. In this case, the header is further allowedto include identification information indicating nighttime or daytime asenvironmental light information, frame counts and so on.

Next, in Step ST40, the output unit 106 integrates two frames andoutputs the frame to the cable 400. That is, when the frame is anodd-number frame, image data of the whole screen obtained in the postprocessing unit 105 is outputted, and when the frame is an even-numberframe, image data of the ¼ reduced screen obtained in the postprocessing unit 105 is outputted.

Next, the in-vehicle camera 100 determines whether the processing endsor not in Step S41. When the processing does not end, the processreturns to Step ST32 and the same process described above is repeated.When the processing ends, the processing is ended immediately in StepS42.

A flowchart of FIG. 13 shows another example of a processing procedureof the camera ECU 200. The flowchart shows the processing procedure withrespect to one frame image data. The camera ECU 200 starts processing inStep ST51. Next, in Step ST52, the input unit 202 takes one frame ofimage data transmitted from the in-vehicle camera 100.

Next, in Step ST53, the image signal processing unit 203 extracts aheader continued from a synchronization signal of the image data. Then,the image signal processing unit 203 determines whether the frame is thewhole screen frame or the ¼ reduced screen frame based on frameinformation included in the header in Step ST54.

When the frame is not the whole screen frame, namely, the frame is the ¼reduced screen frame, the image signal processing unit 203 performsimage recognition processing based on image data of the ¼ reduced screenin Step ST55. In this case, an optimum detection filter is useddepending on nighttime or daytime based on the environmental lightinformation. Additionally, as frame addition and three-dimensional noisereduction processing are performed to the image data of the ¼ reducedscreen as described above, the image signal processing unit 203 performsrecognition processing for a low resolution image.

Next, the image signal processing unit 203 performs correctionprocessing of coordinate information of the recognized image to wholescreen coordinates in Step ST56. When the frame is the whole screen, theimage signal processing unit 203 performs image recognition processingbased on image data of the whole screen in Step ST57. Then, the imagesignal processing unit 203 delivers image recognition results obtainedin Step ST56 and Step ST57, namely, coordinate information of therecognized image with a time stamp to the vehicle control ECU (not shownin FIG. 1) in Step ST58.

Next, the camera ECU 200 determines whether the process ends or not inStep ST59. When the process does not end, the process returns to StepST52 and proceeds with processing of next one frame. On the other hand,when it is determined that the process ends, the process is ended inStep ST60.

As described above, image data of the whole screen and image data of the¼ reduced screen are alternately outputted from the in-vehicle camera100. Accordingly, the image recognition processing using both screensrespectively can be appropriately performed in the camera ECU 200 in thesubsequent stage, which can increase recognition accuracy. As theenvironmental light information is included in the header, parameters ofthe detection filter can be suitably adjusted depending on nighttime ordaytime in the camera ECU 200, therefore, the recognition accuracy canbe increased.

As described above, the in-vehicle camera 100 can sequentially outputframe image data obtained by performing a given number of, for example,two different processing in the imaging processing system 10 shown inFIG. 1. Accordingly, for example, in the camera ECU 200 in thesubsequent stage, these image data can be acquired without performingprocessing of obtaining particular image data such as the camera bird'seye-view conversion image data and the reduced screen image data.Therefore, the processing load of the camera ECU can be reduced and theprocessing can be facilitated.

Also in the imaging processing system 10 shown in FIG. 1, the header atleast including identification information for identifying to whichframe the image data corresponds in the given number of frames to imagedata of each frame outputted from the in-vehicle camera 100.Accordingly, the camera ECU 200 in the subsequent stage can determinewhat type of processing has been performed to image data of each frameaccurately and easily, therefore, adequate processing can be performed.

Also in the imaging processing system 10 shown in FIG. 1, the headeradded to image data of each frame outputted from the in-vehicle camera100 also includes environmental light information and so on, the optimumprocessing can be selectively performed also by using the information inthe camera ECU 200 in the subsequent stage.

<2. Modification Example>

In the above embodiment, the example in which frame image data of thevisual image (normal wide-angle camera image) and frame image data ofthe bird's eye-view conversion image are alternately outputted from thein-vehicle camera 100 and the example in which frame image data of thewhole screen and the frame image data of the reduced screen arealternately outputted are shown. However, the present disclosure is notlimited to the above and it is also preferable that frame image data towhich other different processing is performed is alternately orirregularly outputted. For example, an example in which frame image dataof a color image Icr and frame image data of a monochrome image Iwb arealternately outputted as shown in FIG. 14 can be considered.

The present disclosure may be configured as follows.

(1) An imaging apparatus including

an image sensor,

a signal processing unit repeatedly performing processing different fromeach other in a given number of sequential frames to imaging signalsobtained by the image sensor to thereby obtain image data of respectiveframes, and

an image data output unit sequentially outputting the image data ofrespective frames.

(2) The imaging apparatus described in the above (1), further including

a header addition unit adding a header at least including identificationinformation for identifying to which frame the image data corresponds inthe given number of frames to image data of each frame obtained by thesignal processing unit.

(3) The imaging apparatus described in the above (2),

in which the header addition unit adds the header to an area outside aneffective image area or an area inside the effective image area in eachframe.

(4) The imaging apparatus described in the above (3),

in which the header addition unit adds the header by using part of bitsin a given number of pixel data when adding the header in the areainside the effective image area in each frame.

(5) The imaging apparatus described in any of the above (1) to (4),

in which the signal processing unit obtains first frame image data byperforming first processing in a first frame and obtains second frameimage data by performing second processing in a second frame insequential two frames of the imaging signals obtained by the imagesensor.

(6) The imaging apparatus described in the above (5),

in which the first frame image data is visual image data and the secondframe image data is image data to which image conversion is performed.

(7) The imaging apparatus described in the above (6),

in which the second frame image data is image data to which bird'seye-view conversion is performed.

(8) The imaging apparatus described in the above (6),

in which the image data to which the bird's eye-view conversion isperformed is image data obtained by superimposing a chroma edge signalon a luminance signal.

(9) The imaging apparatus described in the above (5),

in which the first frame image data is normal image data and the secondframe image data is image data obtained by superimposing a chroma edgesignal on a luminance signal.

(10) The imaging apparatus described in the above (5),

in which the first frame image data is image data of a whole screen andthe second frame image data is image data of a reduced screen.

(11) The imaging apparatus described in the above (5),

in which the first frame image data is color image data and the secondframe image data is monochrome image data.

(12) An imaging processing method including

obtaining imaging signals by an image sensor,

repeatedly performing processing different from each other in a givennumber of sequential frames to the imaging signals to thereby obtainimage data of respective frames, and

sequentially outputting the obtained image data of respective frames.

(13) The imaging processing method described in the above (12),

in which an object is imaged with illumination intensity in a wide rangeby performing exposure processing different according to the frame inthe process of obtaining the imaging signals by the image sensor.

(14) An imaging apparatus including

an image sensor, and

an information addition unit adding information of peculiar setting andprocessing to a header portion of a corresponding frame asidentification determination information in accordance with signalprocessing of an imaging signal obtained by the image sensor.

(15) An imaging apparatus including

an image sensor, and

an image processing unit,

in which the image processing unit extracts a chroma edge signal withrespect to part of a taken image obtained by the image sensor andsuperimposes the chroma edge signal on a luminance signal not having aluminance signal boundary in the same direction.

(16) An image processing device including

an image data input unit inputting image data, in which the image datais obtained by repeatedly performing processing different from eachother in a given number of sequential frames, and a header at leastincluding identification information for identifying to which frame theimage data corresponds in the given number of frames is added to imagedata of each frame, and

a processing unit appropriately performing processing of the inputtedimage data of each frame based on the added header.

(17) An imaging processing system including

an imaging apparatus, and

an image processing device,

in which the imaging apparatus includes

an image sensor,

a signal processing unit repeatedly performing processing different fromeach other in a given number of sequential frames to imaging signalsobtained by the image sensor to thereby obtain image data of respectiveframes,

a header addition unit adding a header at least including identificationinformation for identifying to which frame the image data corresponds inthe given number of frames to the image data of each frame, and

an image data output unit outputting the image data of respectiveframes, and

in which the image processing device includes

an image data input unit inputting image data, and

a processing unit appropriately performing processing of the inputtedimage data of each frame based on the added header.

(18) The imaging processing system described in the above (17),

in which the header includes at least one or more information of sensorelectronic shutter, sensor amplifier gain and lens aperture information,lens filter information, temperature information, image differentiationcomponent adjustment parameter values, image gray-scale adjustmentparameters, image resolution adjustment parameters, processing betweenframes adjustment parameters, illumination estimation information,environment estimation information, camera AE adjustment information,camera WB adjustment information, image conversion presence information,image histogram information, image conversion setting information andimage cutout coordinate information.

(19) An imaging processing system having an imaging apparatus and animage processing device, including

a means for outputting images in units of frames by the imagingapparatus,

a means for adding a means for uniquely identifying the frame to aheader of the image to be outputted, and

a selection means for appropriately allocating the outputted images tounique memory addresses based on the image headers.

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2012-099505 filed in theJapan Patent Office on Apr. 25, 2012, the entire contents of which arehereby incorporated by reference.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. An imaging apparatus comprising: signalprocessing circuitry electrically connected to image acquisitioncircuitry, the image acquisition circuitry is configured to convert asignal of an image into sequential frames of image data, wherein thesignal processing circuitry is configured to: electronically receive thesequential frames from the image acquisition circuitry, electronicallyperform first processing on one of the sequential frames when the signalprocessing circuitry determines that said one of the sequential framesis from a first category of frames, the first category differs from asecond category of frames, electronically add first identificationinformation to said one of the sequential frames after the signalprocessing circuitry performs the first processing, the firstidentification information identifies said one of the sequential framesas being from the first category, electronically perform secondprocessing on said one of the sequential frames when the signalprocessing circuitry determines that said one of the sequential framesis from the second category, a field of view of the image after thefirst processing differs from a field of view of the image after thesecond processing, and electronically add second identificationinformation to said one of the sequential frames after the signalprocessing circuitry performs the second processing, the secondidentification information identifies said one of the sequential framesas being from the second category.
 2. The imaging apparatus according toclaim 1, wherein the first category is color image data and the secondcategory is monochrome image data.
 3. The imaging apparatus according toclaim 1, wherein the first category is frame image data of a wholescreen.
 4. The imaging apparatus according to claim 3, wherein the firstcategory is visual image data.
 5. The imaging apparatus according toclaim 3, wherein the second category is either frame image data from aportion of the whole screen or the frame image data of the whole screenthat is reduced in size.
 6. The imaging apparatus according to claim 3,wherein the second category is image data to which image conversion isperformed.
 7. The imaging apparatus according to claim 3, wherein thesecond category is image data to which bird's eye-view conversion isperformed.
 8. The imaging apparatus according to claim 7, wherein theimage data to which the bird's eye-view conversion is performed is imagedata obtained by superimposing a chroma edge signal on a luminancesignal.
 9. The imaging apparatus according to claim 1, wherein the firstcategory is an odd-number frame and the second first category is aneven-number frame.
 10. The imaging apparatus according to claim 1,wherein the image acquisition circuitry is configured to output a framefrom the first category before outputting a frame from the secondcategory.
 11. The imaging apparatus according to claim 10, wherein thesignal processing circuitry is configured to perform the firstprocessing when said one of the sequential frames is the frame from thefirst category.
 12. The imaging apparatus according to claim 10, whereinthe signal processing circuitry is configured to perform the secondprocessing when said one of the sequential frames is the frame from thesecond category.
 13. An image processing device comprising: an inputunit configured to electronically receive sequential frames of imagedata obtained by image acquisition circuitry, a header is in one of thesequential frames; and an image signal processor electrically connectedto the input unit, the image signal processor is configured toelectronically receive the sequential frames from the image acquisitioncircuitry, wherein the image signal processor is configured to:electronically determine whether or not frame information in the headerincludes either first identification information or secondidentification information, the first identification informationidentifies said one of the sequential frames as being from a firstcategory of frames, electronically perform a first procedure on said oneof the sequential frames when the image signal processor determines thatthe frame information includes the first identification information, afield of view of the image after the first procedure differs from afield of view of the image after a second procedure, and electronicallyperform the second procedure on said one of the sequential frames whenthe image signal processor determines that the frame informationincludes the second identification information, the secondidentification information identifies said one of the sequential framesas being from a second category of frames that differs from the firstcategory.
 14. The image processing device according to claim 13, whereinthe header includes information from a group consisting of sensorelectronic shutter information, sensor amplifier gain and lens apertureinformation, lens filter information, temperature information, imagedifferentiation component adjustment parameter values, image gray-scaleadjustment parameters, image resolution adjustment parameters,processing between frames adjustment parameters, illumination estimationinformation, environment estimation information, camera AE adjustmentinformation, camera WB adjustment information, image conversion presenceinformation, image histogram information, image conversion settinginformation and image cutout coordinate information.
 15. The imageprocessing device according to claim 13, wherein the first category iscolor image data and the second category is monochrome image data. 16.The image processing device according to claim 13, wherein the firstcategory is frame image data of a whole screen.
 17. The image processingdevice according to claim 16, wherein the first category is visual imagedata.
 18. The image processing device according to claim 16, wherein thesecond category is either frame image data from a portion of the wholescreen or the frame image data of the whole screen that is reduced insize.
 19. The image processing device according to claim 16, wherein thesecond category is image data to which image conversion is performed.20. The image processing device according to claim 16, wherein thesecond category is image data to which bird's eye-view conversion isperformed.
 21. The image processing device according to claim 20,wherein the image data to which the bird's eye-view conversion isperformed is image data obtained by superimposing a chroma edge signalon a luminance signal.
 22. The image processing device according toclaim 13, wherein the first category is an odd-number frame and thesecond first category is an even-number frame.
 23. An imaging processingsystem comprising: the image processing device according to claim 13;and an imaging apparatus that outputs the sequential frames to the imageprocessing device, the imaging apparatus is configured to:electronically perform first processing on said one of the sequentialframes when signal processing circuitry determines that said one of thesequential frames is from the first category, electronically add thefirst identification information to the header in said one of thesequential frames after the signal processing circuitry performs thefirst processing, electronically perform second processing on said oneof the sequential frames when the signal processing circuitry determinesthat said one of the sequential frames is from the second category,electronically add the second identification information to the headerin said one of the sequential frames after the signal processingcircuitry performs the second processing.
 24. An imaging methodcomprising steps of: electronically, by image acquisition circuitry,converting a signal of an image into sequential frames of image data,wherein the image acquisition circuitry is electrically connected tosignal processing circuitry; electronically, by the signal processingcircuitry, performing first processing on one of the sequential frameswhen signal processing circuitry determines that said one of thesequential frames is from a first category of frames, wherein the firstcategory differs from a second category of frames; electronically, bythe signal processing circuitry, adding first identification informationto said one of the sequential frames after the signal processingcircuitry performs the first processing, wherein the firstidentification information identifies said one of the sequential framesas being from the first category; electronically, by the signalprocessing circuitry, performing second processing on said one of thesequential frames when the signal processing circuitry determines thatsaid one of the sequential frames is from the second category, wherein afield of view of the image after the first processing differs from afield of view of the image after the second processing; andelectronically, by the signal processing circuitry, adding secondidentification information to said one of the sequential frames afterthe signal processing circuitry performs the second processing, whereinthe second identification information identifies said one of thesequential frames as being from the second category.